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ABSTRACT 



J. Barnette and J. McLean (1996) proposed a method of 
controlling Type I error in pairwise multiple comparisons after a significant 
omnibus F test. This procedure, called Alpha-Max, is based on a sequential 
cumulative probability accounting procedure in line with Bonferroni 
inequality. A missing element in the discussion of Alpha-Max was the 
empirical determination of actual probabilities of Type I errors. This paper 
compares the Type I error rates of Alpha-Max with other commonly used 
multiple comparison procedures: (1) Fisher's Least Significant Difference 

(LSD); (2) Dunn-Bonferroni ; (3) Tukey's Honestly Significant Difference 

(HSD) ; (4) the Student Newman Keuls (SNK) procedure; and (5) the Scheffe 

approach. Monte Carlo procedures were used to generate 10,000 replications 
with varied alpha of 0.05 and 0.01; 3, 4, and 5 groups; and 5 sample sizes. 
Actual Type I error rates were determined for the greatest difference and for 
total number of Type I errors. Results indicate that in virtually every 
situation LSD and Alpha-Max had significantly higher probability of Type I 
errors than the other four methods. SNK and HSD had higher than nominal alpha 
probabilities for committing Type I errors, with SNK having a lower level 
than HSD. Dunn-Bonferroni had a level slightly lower than the nominal level, 
while the Scheffe had a level much lower than the nominal level. Varying 
sample size had little practical significance. While Alpha-Max did not 
provide for acceptable experiment -wise control of Type I error, it may 
provide an alternative for control of Type I error in the planned, 
nonorthogonal situation or in situations where assumptions of analysis of 
variance are violated. (Contains 25 tables and 4 references.) (Author/SLD) 
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ABSRACT 



Barnette and McLean (1996, November) proposed a method of 
controlling Type I error in pairwise multiple comparisons after a 
significant omnibus F test. This procedure, called Alpha-Max, 
was based on a sequential cumulative probability accounting 
procedure in line with the Bonferroni inequality. A missing 
element in the discussion of Alpha-Max was the empirical 
determination of actual probabilities of Type I errors. This 
paper compares the Type I error rates of Alpha-Max with five 
other commonly used multiple comparison procedures: Fisher's LSD, 
Dunn-Bonferroni, Tukey's HSD, SNK, and the Scheffe' approach, as 
applied to pairwise differences. Monte Carlo procedures were 
used to generate 10,000 replications, sampling from a unit normal 
population, in every combination of three factors: alpha of .05 
and .01; number of groups of 3, 4, and 5; and sample sizes of 5, 
20, 35, and 50. Actual Type I error rates were determined for 
the greatest difference and for total number of Type I errors. 
These were compared using a two-way design of the multiple 
comparison procedure crossed with sample sizes. This analysis 
was conducted within each combination of alpha and number of 
groups . 

Results indicate that in virtually every situation LSD and 
Alpha-Max had significantly higher probability of Type I errors 
than the other four methods. SNK and HSD had higher than 
nominal alpha probabilities of committing Type I errors, with SNK 
having a higher level than HSD. Dunn-Bonferroni had a level 
slightly lower than the nominal level while the Scheffe had a 
level much lower than the nominal level. Varying sample size had 
little practical significance. While Alpha-Max did not provide 
for acceptable experimentwise control of Type I error, it may 
provide an alternative for control of Type I error in the 
planned, non-orthogonal situation or in situations where 
assumptions of ANOVA are violated. 
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A Comparison of Type I Error Rates of Alpha-Max 
with Established Multiple Comparison Procedures 

A novel and potentially useful new multiple comparison 
procedure (MCP) was introduced in 1996 by Barnette and McLean 
(1996, November) . They referred to this new procedure as Alpha- 
Max. They used selective data to demonstrate Alpha-Max’s 
apparent superiority for controlling Type I error rates to 
Fisher’s Least Significant Difference (LSD) procedure with the 
Bonferroni correction, Tukey’s Honestly Significant Difference 
(HSD) procedure, and the Student Newman Keuls (SNK) procedure. 
However, this was done with data sets purposefully and 
selectively configured to model specific situations. The purpose 
of the present research is to compare empirically determined Type 
I error rates of Alpha -Max, Fisher's LSD procedure, the SNK 
procedure, Tukey's HSD procedure, a pairwise application of the 
Dunn (Bonferroni) procedure, and a pairwise application of the 
Scheffe procedure using Monte Carlo procedures. 

It should be noted that this study is limited to 
consideration of Type I errors. The authors recognize the 
importance of power, effect size, and replication, but since the 
primary purpose of this study is the evaluation of the relatively 
new Alpha-Max procedure, it was limited to the first criteria 
considered by most researchers, the probability of making Type I 
errors . 

Background 

Multiple comparison procedures were developed as followups 
to an omnibus F-test in the analysis of variance (ANOVA) . It 
quickly became apparent that the Type I error rates would be 
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inflated if multiple significance tests without some control were 
used. Type I error rates can be described for a number of 
situations. One situation is where Type I error is described to 
cover a specified number of tests simultaneously. For example, 
an hypothesis-wise error rate refers to the chance of making a 
Type I error for one hypothesis or, in this case, one comparison. 
On the other hand, an experiment-wise Type I error rate refers to 
the chance of making an error for all of the tests in a complete 
experiment or at least for a family of comparisons. In this 
paper, we refer to hypothesis-wise error rate as the probability 
of making a Type I error for one test of a hypothesis and an 
experiment -wise error rate as the probability of making at least 
one Type I error when testing all of the pairwise comparisons 
following a significant omnibus F-test in ANOVA. 

Traditional comparison of the procedures’ control of Type I 
errors, it has been done traditionally using two approaches. The 
first is the comparison of only the largest difference. While 
this approach does provide assistance to a research or trying to 
identify which MCP to use, the purpose for using an MCP is to 
identify all differences. Thus, a more useful approach would be 
to compare the various MCPs in their abilities to control error 
for identifying all pairwise differences. 

As previously noted, the present study will compare Alpha- 
Max, Fisher's LSD procedure, the SNK procedure, Tukey's HSD 
procedure, a pairwise application of the Dunn (Bonferroni) 
procedure, and a pairwise application of the Scheff§ procedure 
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using Type I error rates as the dependent variable. Each of 
these procedures is described briefly below; 

Alpha-Max. The Alpha-Max procedure uses the a priori alpha 
(Type I probability) by ordering the pair-wise difference 
p-values from lowest to highest, accumulating the actual p-values 
from lowest to highest until the addition of the next highest 
p-value exceeds the pre-set nominal alpha (a) level. All pair- 
wise differences whose p-values are already included in the set 
are considered significant, but the one(s) that result (s) in the 
sum of the p-values being higher than the a priori oc is/are not 
considered significant (Barnette & McLean, 1996, November) . 

Fisher’s LSD Procedure. The LSD procedure is eguivalent to 
conducting all pair-wise comparisons using independent t-tests 
with the as the common pooled variance estimate (Kirk, 

1982) . 

SNK Procedure. The SNK uses Student’s t-statistic in a 
layered approach. First the differences between means are 
ordered from largest to smallest differences. The pairs are 
grouped based on their location in the ordered set. Those at 
each extreme are in the first group, those next extreme in the 
second group, and so on until the last group has those adjacent 
to each other in magnitude. A critical value is computed for 
each group using the HSD procedure with the number of groups set 
to the number of means between the ordered pairs including the 
two means being compared (Kirk, 1982) . 

Dunn (Bonferroni) Procedure. The Dunn procedure uses the 
Bonferroni ineguality as authority to divide egually the a priori 




Q 

0 



4 



error among the number of tests to be completed. For example, if 
the a priori a is .05 and 10 tests are required, each test will 
be run at the .005 level (Hayes, 1988). 

Scheffe Procedure. The Scheffe procedure allows for all 
possible comparisons by adjusting the critical F-value. The 
adjustment is to use the F-value for the omnibus F-test 
multiplied by the omnibus degrees of freedom as the critical F- 
value for each comparison (Kirk, 1982) . 

The Dictionary of Statistics and Methodology defines Monte 
Carlo Methods as “any generating of random values (most often 
with a computer) to study statistical models” (Vogt, 1993, p. 

143). Probably its most well-known application was in the 
classic study of ANOVA and ANCOVA assumptions published in the 
Review of Educational Research in 1972 (Glass, Peckham, & 

Sanders) . Monte Carlo techniques combined with the power of 
modern computers has opened a new world of research 
possibilities. Situations that are too complex or cumbersome for 
analytic solutions can be explored empirically using Monte Carlo 
methods. Statistical modeling based on Monte Carlo procedures 
has become common in disciplines ranging from the hard sciences 
to the social sciences. Who cannot relate to the weather 
forecasts that often use computer modeling techniques? Monte 
Carlo methods have become a recognized tool in educational 
research. 



Research Questions 

The following research questions are addressed in this 
study ; 
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1. When a is set at .05, how do the proportions of 
identifying at least one Type I error compare among the 
six multiple comparison procedures when K= 3 , K= 4 , and 
K= 5? 

2. When a is set at .01, how do the proportions of 
identifying at least one Type I error compare among the 
six multiple comparison procedures when K= 3 , K= 4 , and 
K= 5? 

3. When a is set at .05, how do the proportions of total 

Type I errors compare among the six multiple comparison 

procedures when K= 3 , K= 4 , and K= 5? 

4. When a is set at .01, how do the proportions of total 

Type I errors compare among the six multiple comparison 

procedures when K= 3 , K= 4 , and K= 5? 



Procedures 

The basic design used was a procedures (6 levels) by sample 
size (4 levels) factorial design with 10 observations per cell, 
repeated and analyzed separately for six combinations of number 
of groups (3, 4, and 5) and two levels of a (.05 and .01). For 
each situation, there were six multiple comparison procedures 
used to analyze the resulting data sets; LSD, Alpha-Max, Student- 
Newman-Keuls , HSD, Dunn, and Scheffe crossed with four sample 
sizes; 5, 20, 35, and 50 per sample. 

There were two dependent variables; proportion of times 
there was at least one Type I error (significant difference found 
for largest pairwise mean difference) and total proportion of 
Type I errors (total number of significant differences) . 

Each cell had ten observations. These observations were 
determined in the following way; 

1. Sample size (n) , number of groups (K) , and a level were 
set. 



2. Observations (n) were randomly generated from a normal 
distribution of z score deviates for each of the K 
groups. 

3. Means were computed for each group. 

4. These means were compared using one-way ANOVA. 

5. If the omnibus F statistic was significant at the level 
of significance, pairwise differences were compared 
using each of the six multiple comparison procedures. 

6 . When the multiple comparison procedures were used, two 
observations were recorded for each procedure: 
determination of at least one significant difference 
and total number of significant differences. 

7. Steps 1 through 7 were repeated 1000 times. 

8 . Number of times per 1000 at least one significant 
difference was found (for largest pairwise mean 
difference) and total number of significant differences 
were recorded for each multiple comparison procedure; 
these were converted to proportions which became the 
unit of analysis. 

9. Steps 1 through 8 were repeated 10 times using a PC 
program written by the first author (PMMCP) in double- 
precision QBASIC, compiled and run on a Pentium 90, 
Gateway PC. 

10. Data generated by PMMCP were saved as PC data files 
which were then transferred to a mainframe computer for 
final analysis using SAS®. 

11. Steps 1 through 10 were repeated for each group number 
(K= 3, K= 4, and K= 5) crossed with each nominal a (.05 
and . 01 ) . 

For each of the six groups by a combination a two-way ANOVA 
was conducted with multiple comparison procedure crossed with 
sample size. Three effects were examined: the interaction of MCP 
and sample size, and, if not significant, main effect tests were 
conducted for the MCP and sample size independent variables. In 
addition, practical significance was reported using 77 ^ 
(SSeffect/SStotal)- 
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Results 



Results are presented in four sections, each relating to one 
of the research questions. A brief discussion of the results is 
also presented in the four sections. 

At least one Type I error when a= . 05 

Research question 1 is: 

1. When a is set at .05, how do the proportions of 

identifying at least one Type I error compare among the 
six multiple comparison procedures when K= 3 , K= 4 , and 
K= 5? 

The focus here is on finding at least one significant 
difference related to the largest pairwise mean difference. Of 
course, in this case the MCP's are not unique. The LSD and 
Alpha-Max will be the same since, if the largest difference has 
an actual probability less than a, LSD will identify the 
difference as being significant and so will Alpha-Max since the 
accumulated a will also be less than a. Also, since the SNK and 
HSD are the same at the highest number of steps, also the largest 
difference, they will be consistent. Thus, the primary 
differences here are among the LSD/Alpha-Max set, the SNK/HSD 
set, the Dunn, and the Scheffe. 

Results related to this question are presented in Tables 1 
through 6. Table 1 presents the proportion of Type I errors 
found with the largest pairwise mean difference by MCP and sample 
size for three groups, and Table 2 presents the results of the 
two-way ANOVA. There was no significant interaction of MCP and 
sample size. There was a significant main effect for sample size 
difference; however, the practical significance was low (.103). 
There was a significant main effect among the MCP's with a 
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relatively high practical significance (.235). The LSD/Alpha-Max 
MCP had a proportion of .0524, slightly higher than a; while the 
SNK/HSD had a proportion of .0474, slightly lower than a; the 
Dunn MCP had a proportion of .0433; and the Scheffe procedure had 
a proportion of .0407. Follow-up, using the HSD procedure, 
indicated the LSD/Alpha-Max set had a significantly higher 
proportion than any of the other MCP's; the SNK/HSD set had a 
higher proportion than the Scheffe but not higher than the Dunn. 
In the K= 3 situation, LSD and Alpha-Max were liberal, while the 
other methods were conservative as compared with the nominal a 
level . 

Table 3 presents the proportion of Type I errors found with 
the largest pairwise mean difference by MCP and sample size for 
four groups, and Table 4 presents the results of the two-way 
ANOVA. There was no significant interaction of MCP and sample 
size. There was a significant main effect for sample size 
difference; however, the practical significance was low (.071). 
There was a significant main effect among the MCP's with a very 
high practical significance (.545). The LSD/Alpha-Max MCP had a 
proportion of .0476, slightly lower than a, while the SNK/HSD had 
a proportion of .0402, lower than a; the Dunn MCP had a 
proportion of .0349; and the Scheffe procedure had a proportion 
of .0256, both much lower than a. Follow-up, using the HSD 
procedure, indicated the LSD/Alpha-Max set had a significantly 
higher proportion than any of the other MCP's, the SNK/HSD set 
had a higher proportion than the Dunn and Scheffe, and the Dunn 
had a higher proportion than the Scheffe. In the K= 4 situation. 
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all six of the methods were conservative as compared with the 
nominal a level. 

Table 5 presents the proportion of Type I errors found with 
the largest pairwise mean difference by MCP and sample size for 
five groups, and Table 6 presents the results of the two-way 
ANOVA. There was no significant interaction of MCP and sample 
size. There was a significant main effect for sample size 
difference; however, the practical significance was low (.010). 
There was a significant main effect among the MCP's with a very 
high practical significance (.779). The LSD/Alpha-Max MCP had a 
proportion of .0499, essentially at a, while the SNK/HSD had a 
proportion of .0403, lower than a; the Dunn MCP had a proportion 
of .0347; and the Scheffe procedure had a proportion of .0190, 
both much lower than a. Follow-up, using the HSD procedure, 
indicated the LSD/Alpha-Max set had a significantly higher 
proportion than any of the other MCP's, the SNK/HSD set had a 
higher proportion than the Dunn and Scheffe, and the Dunn had a 
higher proportion than the Scheffe. In the K= 5 situation, LSD 
and Alpha-Max were at the nominal a level, while the other four 
methods were conservative as compared with the nominal a level. 

In general, all six of the methods provided good control of 
Type I error for at least one pairwise difference, particularly 
when K was greater than three. LSD, Alpha-Max, SNK, and HSD 
provided proportions relatively close to a, while the Dunn and 
Scheffe approaches were very conservative. There was no 
interaction of MCP and sample size, and while there was a 
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significant main effect for sample size, this was not accompanied 
by practical significance. 

At least one Type I error when a= .01 

Research question 2 is: 

2. When a is set at .01, how do the proportions of 

identifying at least one Type I error compare among the 
six multiple comparison procedures when K= 3, K= 4, and 
K= 5? 

Again, the focus here is on finding at least one significant 
difference related to the largest pairwise mean difference. Of 
course, in this case the MCP's are not unique. The LSD and 
Alpha-Max will be the same, since if the largest difference has 
an actual probability less than a, LSD will identify the 
difference as being significant and so will Alpha-Max because the 
accumulated a will also be less than a. Also, since the SNK and 
HSD are the same at the highest number of steps, also the largest 
difference, they will be consistent. Thus, the primary 
differences here are among the LSD/Alpha-Max set, the SNK/HSD 
set, the Dunn, and the Scheffe. 

Results related to this question are presented in Tables 7 
through 12 . Table 7 presents the proportion of Type I errors 
found with the largest pairwise mean difference by MCP and sample 
size for three groups, and Table 8 presents the results of the 
two-way ANOVA. There was no significant interaction of MCP and 
sample size. There was a significant main effect for sample size 
difference; however, the practical significance was low (.037). 
There was a significant main effect among the MCP's with a 
relatively low practical significance (.089). The LSD/Alpha-Max 
MCP had a proportion of .0103, slightly higher than a; while the 
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SNK/HSD had a proportion of .0093; slightly lower than a, the 
Dunn MCP had a proportion of .0900, and the Scheffe procedure had 
a proportion of .0079. Follow-up, using the HSD procedure, 
indicated the LSD/Alpha-Max set had a significantly higher 
proportion than the Scheffe. In the K= 3 situation, LSD, Alpha- 
Max, SNK, HSD, and Dunn were relatively close to the nominal a, 
while the Scheffe was conservative as compared with the nominal a 
level . 

Table 9 presents the proportion of Type I errors found with 
the largest pairwise mean difference by MCP and sample size for 
four groups, and Table 10 presents the results of the two-way 
ANOVA. There was no significant interaction of MCP and sample 
size. There was a significant main effect for sample size 
difference; however, the practical significance was low (.068). 
There was a significant main effect among the MCP's with a very 
high practical significance (.449). The LSD/Alpha-Max MCP had a 
proportion of .0103, very close to a, while the SNK/HSD had a 
proportion of .0081, slightly lower than a; the Dunn MCP had a 
proportion of .0072; and the Scheffe procedure had a proportion 
of .0045, both much lower than a. Follow-up, using the HSD 
procedure, indicated the LSD/ Alpha-Max set had a significantly 
higher proportion than any of the other MCP's, the SNK/HSD set 
had a higher proportion than the Scheffe, and the Dunn had a 
higher proportion than the Scheffe. In the K= 4 situation, the 
LSD/ Alpha Max procedures were close to the nominal a while the 
other four procedures were conservative, the Scheffe being very 
conservative . 
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Table 11 presents the proportion of Type I errors found with 
the largest pairwise mean difference by MCP and sample size for 
five groups, and Table 12 presents the results of the two-way 
ANOVA. There was no significant interaction of MCP and sample 
size, nor was there a significant main effect for sample size 
difference. There was a significant main effect among the MCP's 
with a very high practical significance (.458). The LSD/Alpha- 
Max MCP had a proportion of .0104, essentially at a, while the 
SNK/HSD had a proportion of .0078, lower than a; the Dunn MCP had 
a proportion of .0068; and the Scheffe procedure had a proportion 
of .0031, both much lower than a. Follow-up, using the HSD 
procedure, indicated the LSD/Alpha-Max set had a significantly 
higher proportion than any of the other MCP's, the SNK/HSD set 
had a higher proportion than the Scheffe, and the Dunn had a 
higher proportion than the Scheffe. In the K= 5 situation, LSD 
and Alpha-Max were very close to the nominal a level, while the 
other four methods were conservative as compared with the nominal 
a level. The Scheffe procedure was very conservative. 

In general, all six of the methods provided good control of 
Type I error for at least one pairwise difference. LSD and 
Alpha-Max provided proportions very close to a, SNK, and HSD 
provided proportions lower than a, while the Dunn and Scheffe 
approaches were very conservative. There was no interaction of 
MCP and sample size, and while there was a significant main 
effect for sample size, this was not accompanied by practical 
significance. 
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If the concern is control of Type I error for having at 
least one in a given experiment, then all six of these procedures 
are reasonable approaches. LSD and Alpha-Max provide for control 
very close to nominal levels, SNK and HSD tend to be slightly 
conservative, Dunn is very conservative, and Scheffe is extremely 
conservative. However, a true experimentwise Type I error 
control needs to be sensitive to the proportion of total number 
of Type I errors that will occur using these six procedures. The 
next sections of this paper deal with this issue. 

Total proportion of Type I errors when a= .05 

Research question 3 is: 

3. When a is set at .05, how do the proportions of total 

Type I errors compare among the six multiple comparison 
procedures when K= 3 , K= 4 , and K= 5? 

The focus here is the total number of Type I errors. In 
this case the MCP's are unique as discussed earlier in this 
paper. In general, it is expected, based on the theoretical 
foundations of these procedures, the order of MCP's on a scale of 
liberal to conservative is: LSD, Alpha-Max, SNK, HSD, Dunn, and 
Scheffe. 

Results related to this question are presented in Tables 13 
through 18. Table 13 presents the proportion of total number of 
Type I errors by MCP and sample size for three groups, and Table 
14 presents the results of the two-way ANOVA. There was no 
significant interaction of MCP and sample size. There was a 
significant main effect for sample size difference; however, the 
practical significance was relatively low (.121). There was a 
significant main effect among the MCP's with very high practical 
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significance (.544). The LSD MCP had a proportion of .0813, the 
Alpha-Max had a proportion of .0734, the SNK had a proportion of 
.0720, and the Dunn had a proportion of .0573, all higher than a. 
The Dunn had a proportion of .0514, slightly higher than a; and 
the Scheffe procedure had a proportion of .0478; slightly lower 
than a. Follow-up, using the HSD procedure, indicated the LSD 
had a significantly higher proportion than the other five MCP's. 
Alpha -Max and SNK were significantly higher than HSD, Dunn, and 
Scheffe. HSD was significantly higher than Scheffe. In the K= 3 
situation, the Dunn MCP resulted in a Type I error rate closest 
to the nominal level. 

Table 15 presents the proportion of total number of Type I 
errors found by MCP and sample size for four groups, and Table 16 
presents the results of the two-way ANOVA. There was no 
significant interaction of MCP and sample size. There was a 
significant main effect for sample size difference; however, the 
practical significance was low (.016). There was a significant 
main effect among the MCP's with a very high practical 
significance (.829). The LSD had a proportion of .1086, and 
Alpha-Max had a proportion of .0950, both very liberal. The SNK 
had a proportion of .0693, also liberal. The HSD had a 
proportion of .0515, slightly higher than a; the Dunn MCP had a 
proportion of .0434, slightly lower than a; and the Scheffe 
procedure had a proportion of .0302, much lower than a. Follow- 
up, using the HSD procedure, indicated the LSD had a 
significantly higher proportion than all of the other MCPs; 
Alpha-Max was higher than SNK, HSD, Dunn, and Scheffe. SNK was 
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higher than HSD, Dunn, and Scheffe. HSD was higher than Dunn and 
Scheffe, and Dunn was higher than Scheffe. In the K= 4 
situation, the Tukey's HSD had an observed Type I error closest 
to a. LSD, Alpha-Max, and SNK were liberal, while Dunn and 
Scheffe were conservative. 

Table 17 presents the proportion of total number of Type I 
errors found by MCP and sample size for five groups, and Table 19 
presents the results of the two-way ANOVA. There was no 
significant interaction of MCP and sample size. There was a 
significant main effect for sample size difference, however the 
practical significance was very low (.006). There was a 
significant main effect among the MCP's with an extremely high 
practical significance (.917). The LSD MCP had a proportion of 
.1627 and the Alpha-Max had a proportion of .1310, both very 
liberal. The SNK had a proportion of .0790, liberal, while the 
HSD was less liberal with a proportion of .0577. The Dunn MCP 
had a proportion of .0466, slightly conservative, and the Scheffe 
procedure had a proportion of .0236, very conservative. Follow- 
up, using the HSD procedure, indicated the LSD had a 
significantly higher proportion than all of the other MCPs; 
Alpha-Max was higher than SNK, HSD, Dunn, and Scheffe. SNK was 
higher than HSD, Dunn, and Scheffe. HSD was higher than Dunn and 
Scheffe, and Dunn was higher than Scheffe. In the K= 5 
situation, the Dunn had an observed Type I error closest to a. 
LSD, Alpha-Max, and SNK were very liberal, HSD was somewhat 
liberal, while the Scheffe was very conservative. 
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Total proportion of Type I errors when g= . 01 



Research question 4 is: 

4. When a is set at .01, how do the proportions of total 

Type I errors compare among the six multiple comparison 
procedures when K= 3 , K= 4 , and K= 5? 

Again, the focus here is the total number of Type I errors. 
In general, it is expected, based on the theoretical foundations 
of these procedures, the order of MCP's on a scale of liberal to 
conservative is: LSD, Alpha-Max, SNK, HSD, Dunn, and Scheffe. 

Results related to this question are presented in Tables 19 
through 24. Table 19 presents the proportion of total number of 
Type I errors by MCP and sample size for three groups, and Table 
20 presents the results of the two-way ANOVA. There was no 
significant interaction of MCP and sample size. There was a 
significant main effect for sample size difference; however, the 
practical significance was relatively low (.041). There was a 
significant main effect among the MCP's with relatively high 
practical significance (.216). The LSD MCP had a proportion of 
.0138, the Alpha-Max had a proportion of .0129, the SNK had a 
proportion of .0121. The HSD had a proportion of .0105, slightly 
higher than a. The Dunn had a proportion of .0101, very close to 
the nominal a, and the Scheffe procedure had a proportion of 
.0088, slightly lower than a. Follow-up, using the HSD 
procedure, indicated the LSD and Alpha-Max had significantly 
higher proportions than HSD, Dunn, and Scheffe, but not higher 
than SNK. SNK was significantly higher than Scheffe. In the K= 

3 situation, the Dunn MCP resulted in a Type I error rate closest 
to the nominal level. However, HSD was very close. 
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Table 21 presents the proportion of total number of Type I 
errors found by MCP and sample size for four groups, and Table 22 
presents the results of the two-way ANOVA. There was no 
significant interaction of MCP and sample size. There was a 
significant main effect for sample size difference; however, the 
practical significance was low (.037). There was a significant 
main effect among the MCP's with a very high practical 
significance (.645). The LSD had a proportion of .0190, and 
Alpha-Max had a proportion of .0174, both much higher than a. 

The SNK had a proportion of .0113, slightly higher than a. The 
HSD had a proportion of .0098, very close to a. The Dunn MCP had 
a proportion of .0083, slightly lower than a, and the Scheffe 
procedure had a proportion of .0049, much lower than a. Follow- 
up, using the HSD procedure, indicated the LSD and Alpha-Max had 
significantly higher proportions than SNK, HSD, Dunn, and 
Scheffe. SNK was higher than Dunn and Scheffe. HSD and Dunn 
were higher than Scheffe. In the K= 4 situation, the Tukey's HSD 
had an observed Type I error closest to a. LSD, Alpha-Max, and 
SNK were liberal, while Dunn and Scheffe were conservative. 

Table 23 presents the proportion of total number of Type I 
errors found by MCP and sample size for five groups, and Table 24 
presents the results of the two-way ANOVA. There was no 
significant interaction of MCP and sample size, nor was there a 
significant main effect for sample size difference. There was a 
significant main effect among the MCP's with a very high 
practical significance (.626). The LSD MCP had a proportion of 
.0261 and the Alpha-Max had a proportion of .0224, both very 
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liberal. The SNK had a proportion of .0126, somewhat higher than 
a. The HSD had a proportion of .0103, very close to a. The Dunn 
MCP had a proportion of .0086, slightly conservative, and the 
Scheffe procedure had a proportion of .0036, very conservative. 
Follow-up, using the HSD procedure, indicated the LSD and Alpha- 
Max had significantly higher proportions than SNK, HSD, Dunn, and 
Scheffe. SNK was higher than Dunn and Scheffe. HSD and Dunn 
were higher than Scheffe. In the K= 5 situation, the Tukey's HSD 
had an observed Type I error closest to a. LSD, Alpha-Max, and 
SNK were liberal, while Dunn and Scheffe were conservative. 
Summary of Total Type I Error Proportions 

Table 25 summarizes the observed Type I error rates for the 
six MCP's for the combinations of group size and a level. These 
results indicate that only two of the procedures provide 
relatively accurate control of Type I error on an experimentwise 
basis: Tukey's HSD and the Dunn procedure. Of these, the Tukey 
tends to be a little liberal and the Dunn tends to be a little 
conservative. Clearly, LSD and Alpha-Max do not provide 
reasonable control of Type I error; and SNK is less liberal 
compared with these, but still has a higher Type I error rate 
than the nominal a. 

It is interesting to note that total Type I error rates for 
LSD and Alpha-Max increase with an increase in the number of 
groups. Of course, with more groups, there are more pairs which 
may possibly be significant. It would be interesting to 
determine whether this increase is a function of the number of 
pairwise comparisons or a function of the procedures themselves. 
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The Scheffe Type I error rate becomes more conservative as the 
number of groups increases. 

These results indicate that Alpha-Max does not appear to be 
a viable procedure to use for pairwise follow-up. It may, 
however, be an alternative to use for planned comparisons. 

Summary and Recommendations 

When the researcher’s goal was to examine the largest 
difference, all six MCPs tended to provide reasonable control of 
Type I errors. However, Alpha-Max and LSD along with the SNK and 
HSD tended to produce empirical a's most closely resembling the 
nominal a's. The Dunn and Scheffe tended to be overly 
conservative. However, as previously noted, this scenario is not 
likely to be important to most researchers. Most researchers 
would be interested in controlling the total Type I error. 

The results when comparing the total Type I error were 
somewhat different. The HSD and Dunn tend to provide the best 
estimates with HSD results being slightly liberal and Dunn 
results being slightly conservative. The LSD and Alpha-Max 
clearly do not provide reasonable control for Type I error. 
Scheffe results are conservative as always and tend to become 
more so as the number of groups increase. 

While the results do not provide a clear mandate for 
selection of one procedure in every situation, some 
generalizations can be made. The Alpha-Max procedure and the LSD 
procedure tend to provide similar results. The Scheffe procedure 
tends to be quite conservative in most situations. On the 
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average, the HSD and Dunn procedures tend to provide empirical 
a's closest to the nominal a's, but HSD tends to be slightly 
liberal while the Dunn tends to be slightly conservative. 

Finally, the study does not provide support for using the 
Alpha-Max in place of some of the more traditional MCPs for post 
hoc multiple comparison procedures. The initial idea in the 
thinking for the Alpha-Max was that it could provide a more 
powerful a priori MCP. This still may be true and research 
should be conducted to determine if it is. Alpha-Max remains an 
alternative in the sense that it is still probably the easiest to 
implement. The jury remains out on its use as an a priori 
procedure. In addition, it's vaibility as an alternative when 
assumptions such as homogeneity of variance with unequal group 
sizes is to be assessed. 
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Table 1. Observed Type I Error Proportion for First Type I 

Error, Multiple Comparison Procedure by Sample Size, 
K= 3, a= .05, 10 Replications of 1000 Each 



Sample Size 



Multiple 

Comparison 

Procedure 


n= 5 


n= 20 


n= 35 


n= 50 


Total 


LSD 


.0564 


.0496 


.0527 


. 0508 


.0524 


Alpha-Max 


.0564 


.0496 


.0527 


. 0508 


. 0524 


Newman-Keuls 


. 0523 


.0441 


. 0465 


.0466 


.0474 


HSD 


.0523 


.0441 


.0465 


.0466 


.0474 


Dunn 


.0470 


.0405 


.0432 


.0425 


.0433 


Schef fe 


.0465 


. 0363 


. 0404 


.0397 


. 0407 


Total 


.0518 


.0440 


. 0470 


. 0462 


.0473 



Table 2. ANOVA Summary Table, First Type I Error, Multiple 
Comparison Procedure by Sample Size, K= 3, a= .05, 
10 Replications of 1000 Each 



Source 


df 


MS* 


F 


P 




Procedure 


5 


8.8592 


15.41 


.0001 


.235 


Sample size 


3 


6.4875 


11.28 


.0001 


. 103 


Interaction 


15 


0.0488 


0.08 


1.000 


.004 


Error 


216 


0.5750 









Total 239 



* X 10-^ 

Procedure Pairwise Differences: 

AMx SNK HSD Dun Sch 

LSD * * * * 

Amx * * * * 

SNK * 

HSD * 

Dun 
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Table 3 . Observed Type I Error Proportion for First Type I 

Error, Multiple Comparison Procedure by Sample Size, 
K= 4, a= .05, 10 Replications of 1000 Each 



Sample Size 



Multiple 

Comparison 

Procedure 


n= 5 


n= 20 


n= 35 


n= 50 


Total 


LSD 


.0459 


.0434 


.0507 


. 0502 


. 0476 


Alpha-Max 


. 0459 


.0434 


. 0507 


. 0502 


. 0476 


Newman-Keuls 


. 0390 


.0352 


.0448 


.0419 


. 0402 


HSD 


. 0390 


.0352 


.0448 


.0419 


. 0402 


Dunn 


.0326 


.0318 


.0379 


.0374 


.0349 


Schef fe 


.0275 


. 0232 


.0255 


. 0261 


.0256 


Total 


.0383 


.0354 


.0424 


.0413 


.0393 



Table 4. ANOVA Summary Table, First Type I Error, Multiple 
Comparison Procedure by Sample Size, K= 4, a= .05, 
10 Replications of 1000 Each 



Source 


df 


MS* 


F 


P 


^2 

V 


Procedure 


5 


27.6274 


63.72 


. 0001 


. 545 


Sample size 


3 


5.9949 


13.83 


. 0001 


.071 


Interaction 


15 


0.2312 


0.53 


.9202 


. 014 


Error 


216 


0.4336 









Total 239 

* X 10-^ 

Procedure Pairwise Differences; 

AMx SNK HSD Dun Sch 

LSD * * * * 

Amx * * * * 

SNK * * 

HSD * * 

Dun * 
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Table 5. Observed Type I Error Proportion for First Type I 

Error, Multiple Comparison Procedure by Sample Size, 
K= 5, a= .05, 10 Replications of 1000 Each 



Sample Size 



Multiple 

Comparison 

Procedure 


n= 5 


n= 20 


n= 35 


n= 50 


Total 


LSD 


.0514 


.0513 


.0471 


. 0498 


.0499 


Alpha-Max 


.0514 


.0513 


.0471 


.0498 


. 0499 


Newman-Keuls 


.0423 


.0394 


.0392 


.0402 


. 0403 


HSD 


.0423 


.0394 


.0392 


.0402 


.0403 


Dunn 


.0352 


.0355 


.0334 


.0350 


.0347 


Schef fe 


.0220 


.0180 


.0181 


.0179 


.0190 


Total 


.0408 


.0391 


.0374 


.0388 


.0390 



Table 6. ANOVA Summary Table, First Type I Error, Multiple 
Comparison Procedure by Sample Size, K= 5, a= .05, 
10 Replications of 1000 Each 



Source 


df 


MS* 


F 


P 




Procedure 


5 


52.7835 


163.17 


.0001 


.779 


Sample size 


3 


1.1763 


3.64 


.0137 


.010 


Interaction 


15 


0.1001 


0.31 


.9942 


.004 


Error 


216 


0.3235 









Total 239 



* X 10-^ 

Procedure Pairwise Differences: 

AMx SNK HSD Dun Sch 

LSD * * * * 

AMx * * * * 

SNK * * 

HSD * * 

Dun * 




CO 
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Table 7. Observed Type I Error Proportion for First Type I 

Error, Multiple Comparison Procedure by Sample Size, 
K= 3, a= .01, 10 Replications of 1000 Each 



Sample Size 



Multiple 

Comparison 

Procedure 


n= 5 


n= 20 


n= 35 


n= 50 


Total 


LSD 


. 0105 


.0111 


. 0099 


. 0096 


.0103 


Alpha-Max 


. 0105 


.0111 


. 0099 


. 0096 


. 0103 


Newman-Keuls 


. 0096 


. 0099 


. 0091 


. 0085 


. 0093 


HSD 


. 0096 


.0099 


.0091 


. 0085 


. 0093 


Dunn 


. 0089 


. 0096 


.0091 


. 0083 


.0090 


Schef fe 


. 0085 


. 0084 


. 0077 


.0071 


. 0079 


Total 


. 0096 


.0100 


.0091 


. 0086 


.0093 



Table 8. ANOVA Summary Table, First Type I Error, Multiple 
Comparison Procedure by Sample Size, K= 3 , a= .01, 
10 Replications of 1000 Each 



Source 


df 


MS* 


F 


P 




Procedure 


5 


0.3114 


4.40 


.0008 


. 089 


Sample size 


3 


0.2187 


3.09 


. 0279 


.037 


Interaction 


15 


0.0034 


0.05 


1.0000 


.000 


Error 


216 


0.0707 









Total 239 



*x 10-^ 

Procedure Pairwise Differences: 

AMx SNK HSD Dun Sch 
LSD * 

AMx * 

SNK 

HSD 

Dun 



O 
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Table 9. Observed Type I Error Proportion for First Type I 

Error, Multiple Comparison Procedure by Sample Size, 
K= 4, a= .01, 10 Replications of 1000 Each 



Sample Size 



Multiple 

Comparison 

Procedure 


n= 5 


n= 20 


n= 35 


n= 50 


Total 


LSD 


. 0097 


. 0093 


. 0108 


.0112 


. 0103 


Alpha-Max 


. 0097 


. 0093 


. 0108 


.0112 


. 0103 


Newman-Keuls 


. 0069 


. 0072 


.0091 


. 0093 


. 0081 


HSD 


. 0069 


. 0072 


. 0091 


. 0093 


. 0081 


Dunn 


.0060 


. 0067 


. 0079 


.0080 


. 0072 


Schef f e 


. 0045 


. 0044 


.0041 


.0050 


. 0045 


Total 


. 0073 


. 0074 


. 0086 


.0090 


.0081 



Table 10. ANOVA Summary Table, First Type I Error, Multiple 
Comparison Procedure by Sample Size, K= 4, a= .01, 
10 Replications of 1000 Each 



Source 


df 


MS* 


F 


P 




Procedure 


5 


1.8482 


41.56 


.0001 


. 449 


Sample size 


3 


0.4639 


10.43 


. 0001 


.068 


Interaction 


15 


0.0234 


0.53 


.9246 


. 017 


Error 


216 


0.0445 









Total 239 



* X 10-^ 

Procedure Pairwise Differences: 

AMx SNK HSD Dun Sch 
* * * * 

* * * * 

* 

* 

* 



O 




LSD 

AMx 

SNK 

HSD 

Dun 



£8 



Table 11. Observed Type I Error Proportion for First Type I 

Error, Multiple Comparison Procedure by Sample Size, 
K= 5, a= .01, 10 Replications of lOOO Each 



Sample Size 



Multiple 

Comparison 

Procedure 


n= 5 


n= 20 


n= 35 


n= 50 


Total 


LSD 


.0100 


. 0111 


.0101 


. 0102 


. 0104 


Alpha-Max 


.0100 


.0111 


.0101 


. 0102 


.0104 


Newman-Keuls 


.0080 


. 0082 


. 0073 


. 0076 


. 0078 


HSD 


.0080 


. 0082 


. 0073 


. 0076 


. 0078 


Dunn 


. 0068 


. 0078 


. 0061 


. 0066 


. 0068 


Schef fe 


.0037 


. 0031 


. 0025 


. 0029 


. 0031 


Total 


. 0078 


. 0083 


. 0072 


. 0075 


. 0077 



Table 12. ANOVA Summary Table, First Type I Error, Multiple 
Comparison Procedure by Sample Size, K= 5, a= .01, 
10 Replications of lOOO Each 



Source 


df 


MS* 


F 


P 




Procedure 


5 


2.9155 


37.58 


.0001 


.458 


Sample size 


3 


0.1112 


1.43 


.2342 


.010 


Interaction 


15 


0.0097 


0.13 


1.0000 


. 005 


Error 


216 


0.0776 









Total 239 



* X 10-^ 

Procedure Pairwise Differences: 

AMx SNK HSD Dun Sch 
* * * * 

* * * * 

* 

* 

* 



O 




LSD 

AMx 

SNK 

HSD 

Dun 



27 



29 



Table 13. Observed Type I Error Proportion for Total Type I 

Errors, Multiple Comparison Procedure by Sample Size, 
K= 3, a= .05, 10 Replications of 1000 Each 



Sample Size 



Multiple 

Comparison 

Procedure 


n= 5 


n= 20 


n= 35 


n= 50 


Total 


LSD 


. 0923 


. 0755 


.0785 


. 0789 


. 0813 


Alpha-Max 


. 0823 


. 0681 


.0712 


. 0709 


. 0734 


Newman-Keuls 


. 0846 


. 0652 


.0671 


. 0709 


. 0720 


HSD 


. 0659 


. 0522 


.0539 


. 0571 


. 0573 


Dunn 


. 0584 


. 0471 


.0496 


. 0503 


. 0514 


Schef fe 


. 0574 


. 0420 


.0455 


. 0464 


. 0478 


Total 


. 0736 


. 0584 


.0610 


. 0624 


. 0638 



Table 14. ANOVA Summary Table, Total Type I Errors, Multiple 
Comparison Procedure by Sample Size, K= 3, a= .05, 
10 Replications of 1000 Each 



Source 


df 


MS* 


F 


P 


^2 

V 


Procedure 


5 


73.3315 


71.18 


. 0001 


.544 


Sample size 


3 


27.2663 


26.46 


. 0001 


. 121 


Interaction 


15 


0.2148 


0.21 


.9994 


. 005 


Error 


216 


1.0303 








Total 


239 










* X 10-^ 












Procedure Pairwise Differences: 








AMx SNK 
LSD * * 

AMx 
SNK 
HSD 


HSD Dun 

Hr Hr 

Hr Hr 

Hr Hr 


Sch 

Hr 

Hr 

Hr 

Hr 









Dun 



O 




28 



30 



Table 15. Observed Type I Error Proportion for Total Type I 

Errors, Multiple Comparison Procedure by Sample Size, 
K= 4, a= .05, 10 Replications of 1000 Each 



Sample Size 



Multiple 

Comparison 

Procedure 


n= 5 


n= 20 


n= 35 


n= 50 


Total 


LSD 


. 1116 


.0988 


.1117 


. 1122 


. 1086 


Alpha-Max 


. 0970 


. 0868 


. 0974 


. 0987 


. 0950 


Newman-Keuls 


. 0733 


. 0611 


. 0725 


. 0703 


. 0693 


HSD 


. 0532 


.0445 


. 0558 


. 0526 


. 0515 


Dunn 


. 0437 


. 0397 


. 0450 


. 0450 


. 0434 


Schef fe 


. 0348 


. 0275 


.0291 


.0294 


. 0302 


Total 


. 0689 


. 0597 


. 0686 


. 0680 


. 0663 



Table 16. ANOVA Summary Table, Total Type I Errors, Multiple 
Comparison Procedure by Sample Size, K= 4 , a= .05, 
10 Replications of 1000 Each 



Source 


df 


MS* 


F 


P 




Procedure 


5 


373.3313 


235.46 


. 0001 


.829 


Sample size 


3 


11.6544 


7.35 


. 0001 


.016 


Interaction 


15 


0.5595 


0.35 


.9883 


. 004 


Error 


216 


1.5855 









Total 239 



* X 10-^ 

Procedure Pairwise Differences: 



LSD 

AMx 

SNK 

HSD 

Dun 



AMx SNK HSD Dun Sch 
***** 

* * * * 

* * * 

■k k 
k 
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Table 17. Observed Type I Error Proportion for Total Type I 

Errors, Multiple Comparison Procedure by Sample Size, 
K= 5, a= .05, 10 Replications of 1000 Each 



Sample Size 



Multiple 

Comparison 

Procedure 


n= 5 


n= 20 


n= 35 


n= 50 


Total 


LSD 


. 1766 


.1655 


. 1497 


. 1590 


. 1627 


Alpha-Max 


. 1379 


.1340 


. 1222 


. 1299 


. 1310 


Newman-Keuls 


.0839 


.0785 


.0754 


. 0782 


. 0790 


HSD 


. 0609 


.0567 


. 0550 


. 0583 


.0577 


Dunn 


.0462 


.0471 


.0447 


.0482 


.0466 


Schef fe 


.0275 


.0227 


. 0220 


. 0220 


. 0236 


Total 


. 0888 


.0841 


. 0782 


. 0826 


.0834 



Table 18. ANOVA Summary Table, Total Type I Errors, Multiple 
Comparison Procedure by Sample Size, K= 5, a= .05, 
10 Replications of 1000 Each 



Source 


df 


MS* 


F 


P 




Procedure 


5 


1133.8206 


544.26 


. 0001 


.917 


Sample size 


3 


11.6028 


5.57 


.0011 


. 006 


Interaction 


15 


1.7025 


0.82 


. 6579 


. 004 


Error 


216 


2.0832 









Total 239 



* X 10-^ 

Procedure Pairwise Differences: 



LSD 

AMx 

SNK 

HSD 

Dun 



AMx SNK HSD Dun Sch 
***** 

* * * * 

* * * 

* * 

* 



O 

ERIC 
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Table 19. Observed Type I Error Proportion for Total Type I 

Errors, Multiple Comparison Procedure by Sample Size, 
K= 3 , a= .01, 10 Replications of 1000 Each 



Sample Size 



Multiple 

Comparison 

Procedure 


n= 5 


n= 20 


n= 35 


n= 50 


Total 


LSD 


.0155 


. 0138 


.0127 


. 0130 


. 0138 


Alpha-Max 


. 0148 


.0131 


.0119 


.0119 


. 0129 


Newman-Keuls 


. 0139 


.0117 


.0116 


. 0112 


.0121 


HSD 


. 0111 


. 0106 


. 0105 


. 0097 


. 0105 


Dunn 


. 0102 


. 0103 


. 0104 


. 0093 


. 0101 


Schef fe 


. 0097 


.0088 


.0088 


. 0079 


. 0088 


Total 


. 0125 


.0114 


. 0110 


. 0105 


. 0114 



Table 20. ANOVA Summary Table, Total Type I Errors, Multiple 
Comparison Procedure by Sample Size, K= 3, a= .01, 
10 Replications of 1000 Each 



Source 


df 


MS* 


F 


P 




Procedure 


5 


0.1579 


12.85 


. 0001 


.216 


Sample size 


3 


0.0502 


4 . 09 


. 0076 


. 041 


Interaction 


15 


0.0035 


0.28 


.9965 


. 014 


Error 


216 


0.0123 









Total 239 



* X 10-" 

Procedure Pairwise Differences: 

AMx SNK HSD Dun Sch 
LSD * * * 

AMx * * * 

SNK * 

HSD 

Dun 
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Table 21. Observed Type I Error Proportion for Total Type I 

Errors, Multiple Comparison Procedure by Sample Size, 
K= 4, a= .01, 10 Replications of 1000 Each 



Sample Size 



Multiple 

Comparison 

Procedure 


n= 5 


n= 20 


n= 35 


n= 50 


Total 


LSD 


. 0202 


.0163 


. 0204 


. 0190 


. 0190 


Alpha-Max 


.0177 


.0147 


.0194 


. 0176 


. 0174 


Newman-Keuls 


. 0103 


.0089 


.0143 


. 0118 


. 0113 


HSD 


. 0085 


.0083 


. 0116 


. 0106 


. 0098 


Dunn 


. 0070 


.0074 


. 0097 


. 0091 


. 0083 


Schef fe 


. 0050 


.0048 


. 0045 


. 0054 


. 0049 


Total 


.0115 


.0101 


. 0133 


. 0123 


. 0118 



Table 22. ANOVA Summary Table, Total Type I Errors, Multiple 
Comparison Procedure by Sample Size, K= 4, a= .01, 
10 Replications of 1000 Each 



Source 


df 


MS* 


F 


P 


V 


Procedure 


5 


11.6977 


93.20 


. 0001 


. 645 


Sample size 


3 


1.1253 


8.97 


. 0001 


. 037 


Interaction 


15 


0.1179 


0.94 


.5213 


. 019 


Error 


216 


0.1255 









Total 239 



* X 10-^ 

Procedure Pairwise Differences: 

AMx SNK HSD Dun Sch 

LSD * * * * 

AMx * * * * 

SNK * * 

HSD * 

Dun * 




C) 



4 
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Table 23. Observed Type I Error Proportion for Total Type I 

Errors, Multiple Comparison Procedure by Sample Size, 
K= 5, o= .01, 10 Replications of 1000 Each 



Sample Size 



Multiple 

Comparison 

Procedure 


n= 5 


n= 20 


n= 35 


n= 50 


Total 


LSD 


. 0285 


. 0266 


.0257 


. 0235 


. 0261 


Alpha-Max 


. 0243 


. 0231 


. 0214 


. 0206 


. 0224 


Newman-Keuls 


. 0152 


. 0122 


.0117 


. 0114 


. 0126 


HSD 


. 0109 


. 0108 


. 0096 


. 0098 


. 0103 


Dunn 


. 0091 


. 0098 


. 0076 


. 0077 


. 0086 


Schef fe 


. 0045 


. 0038 


. 0028 


. 0031 


. 0036 


Total 


. 0154 


.0144 


.0131 


. 0127 


. 0139 



Table 24. ANOVA Summary Table, Total Type I Errors, Multiple 
Comparison Procedure by Sample Size, K= 5, a= .01, 
10 Replications of 1000 Each 



Source 


df 


MS* 


F 


P 




Procedure 


5 


29.6115 


75.63 


. 0001 


. 626 


Sample size 


3 


0.9204 


2 . 35 


. 0733 


. 012 


Interaction 


15 


0.0625 


0.16 


.9999 


. 004 


Error 


216 


0.3915 









Total 239 



* X 10-^ 

Procedure Pairwise Differences: 

AMx SNK HSD Dun Sch 

LSD * * * * 

AMx * * * * 

SNK * * 

HSD * 

Dun * 







33 



Table 25. Observed Type I Error Proportion for Total Type I 
Errors, Total for Each Alpha Level 





a= .05 


O 

11 


Multiple 

Comparison 

Procedure 


K= 3 


11 


K= 5 


K= 3 


11 


K= 5 


LSD 


. 0813 


. 1086 


. 1627 


. 0138 


. 0190 


. 0261 


Alpha-Max 


.0734 


.0950 


.1310 


.0129 


.0174 


. 0224 


Newman-Keuls 


. 0720 


.0693 


.0790 


.0121 


. 0113 


. 0126 


HSD 


.0573 


.0515 


.0577 


.0105 


. 0098 


. 0103 


Dunn 


.0514 


.0434 


.0466 


.0101 


. 0083 


.0086 


Schef fe 


.0478 


. 0302 


.0236 


.0088 


. 0049 


. 0036 
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